Skip to content

support revealing uses of opaques #139587

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

lcnr
Copy link
Contributor

@lcnr lcnr commented Apr 9, 2025

the following now works, needs a bit of cleanup to actually land :3

fn foo(x: &u32) -> impl Sized + '_ {
    let _ = || {
        let temp = 1;
        // normalization of the return type results
        // in `opaque<'local1> = &'local2 u32`.
        foo(&temp);
    };
    x
}

final breakage?

fn proj<F: FnOnce() -> P, P: Iterator<Item = Q>, Q>(_: F) {}
struct Lts<'a, 'b>(&'a (), &'b ());
fn expectations<'a>() -> impl Iterator<Item = Lts<'a, 'static>> + use<'a> {
    // opaque<'a> = Map<std::iter::Empty<Lts<'0, '1>>, {closure@src/main.rs:5:28: 5:31}>``
    proj::<_, _, Lts<'a, 'static>>(|| expectations());
    std::iter::empty().map(|x| x)
}
struct Foo<F, T>(T, F);
trait Trait {
    type Assoc;
}
impl<F: FnOnce(T) -> R, T, R> Trait for Foo<F, T> {
    type Assoc = R;
}

fn new_defining_use<F: FnOnce() -> R, R>(f: F) {}
fn rpit<'a>() -> impl Trait<Assoc = &'static ()> + use<'a> {
    new_defining_use(rpit::<'a>);
    Foo(&(), |x: &'static ()| x)
}
use std::convert::identity;
struct Foo<F, T>(T, F);
trait Trait {
    type Assoc;
}
impl<F: FnOnce(T) -> R, T, R> Trait for Foo<F, T> {
    type Assoc = R;
}

fn new_defining_use<F: FnOnce() -> R, R>(_: F) {}
fn rpit<'a>() -> impl Trait<Assoc = &'static ()> + use<'a> {
    new_defining_use(rpit::<'a>);
    Foo(&(), identity as fn(_) -> _)
}
struct Inv<'a, 'b>(*mut (&'a (), &'b ()));
fn new_defining_use<F: FnOnce() -> R, R>(_: F) {}
fn rpit<'a>() -> impl Sized + use<'a> {
    new_defining_use(rpit::<'a>);
    Inv::<'a, 'static>(std::ptr::null_mut())
}
fn new_defining_use<F: FnOnce(T) -> R, T, R>(_: F) {}
fn rpit<'a,  'b: 'b>(x: &'b ()) -> impl Sized + use<'a, 'b> {
    new_defining_use(rpit::<'a, 'b>);
    x
}

r? @compiler-errors

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels Apr 9, 2025
@rust-log-analyzer

This comment has been minimized.

@lcnr lcnr added the WG-trait-system-refactor The Rustc Trait System Refactor Initiative (-Znext-solver) label Apr 9, 2025
bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 9, 2025
[DO NOT MERGE] bootstrap with `-Znext-solver=globally`

A revival of rust-lang#124812.

Current status:

~~`./x.py b --stage 2` passes 🎉~~

`try` builds succeed 🎉 🎉 🎉

[first perf run](rust-lang#133502 (comment)) 👻

### in-flight changes

- rust-lang#124852, unsure whether I actually want to land this PR for now
- rust-lang#139587
- https://github.com/lcnr/rust/tree/opaque-type-method-call

r? `@ghost`
@lcnr lcnr force-pushed the revealing-use-closures branch from f83cf4b to b796696 Compare April 10, 2025 07:26
@rust-log-analyzer

This comment has been minimized.

bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 10, 2025
[DO NOT MERGE] bootstrap with `-Znext-solver=globally`

A revival of rust-lang#124812.

Current status:

~~`./x.py b --stage 2` passes 🎉~~

`try` builds succeed 🎉 🎉 🎉

[first perf run](rust-lang#133502 (comment)) 👻

### in-flight changes

- rust-lang#124852, unsure whether I actually want to land this PR for now
- rust-lang#139587
- https://github.com/lcnr/rust/tree/opaque-type-method-call

r? `@ghost`
@lcnr
Copy link
Contributor Author

lcnr commented Apr 10, 2025

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Apr 10, 2025
bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 10, 2025
support revealing uses of opaques in closures

based on rust-lang#139484

the following now works, needs **a bit** of cleanup to actually land :3
```rust
fn foo(x: &u32) -> impl Sized + '_ {
    let _ = || {
        let temp = 1;
        // normalization of the return type results
        // in `opaque<'local1> = &'local2 u32`.
        foo(&temp);
    };
    x
}
```

r? `@compiler-errors`
@bors
Copy link
Collaborator

bors commented Apr 10, 2025

⌛ Trying commit 22d34c4 with merge 14ea441...

@rust-log-analyzer

This comment has been minimized.

@bors
Copy link
Collaborator

bors commented Apr 10, 2025

☀️ Try build successful - checks-actions
Build commit: 14ea441 (14ea4416942169f85851de68119f43a59ef8801f)

@rust-timer

This comment has been minimized.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (14ea441): comparison URL.

Overall result: ❌✅ regressions and improvements - please read the text below

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

Next Steps: If you can justify the regressions found in this try perf run, please indicate this with @rustbot label: +perf-regression-triaged along with sufficient written justification. If you cannot justify the regressions please fix the regressions and do another perf run. If the next run shows neutral or positive results, the label will be automatically removed.

@bors rollup=never
@rustbot label: -S-waiting-on-perf +perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

mean range count
Regressions ❌
(primary)
0.6% [0.2%, 1.1%] 62
Regressions ❌
(secondary)
0.5% [0.3%, 0.8%] 16
Improvements ✅
(primary)
-0.3% [-0.3%, -0.2%] 2
Improvements ✅
(secondary)
-0.5% [-1.1%, -0.2%] 27
All ❌✅ (primary) 0.5% [-0.3%, 1.1%] 64

Max RSS (memory usage)

Results (primary 2.0%, secondary -1.3%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
2.0% [1.2%, 2.8%] 5
Regressions ❌
(secondary)
3.1% [2.5%, 3.6%] 2
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-3.0% [-7.2%, -1.2%] 5
All ❌✅ (primary) 2.0% [1.2%, 2.8%] 5

Cycles

Results (primary 1.1%, secondary 1.5%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
1.1% [0.8%, 1.3%] 12
Regressions ❌
(secondary)
1.5% [1.2%, 1.9%] 10
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) 1.1% [0.8%, 1.3%] 12

Binary size

Results (secondary -0.1%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-0.1% [-0.1%, -0.1%] 6
All ❌✅ (primary) - - 0

Bootstrap: 780.245s -> 782.612s (0.30%)
Artifact size: 366.19 MiB -> 366.30 MiB (0.03%)

@rustbot rustbot added perf-regression Performance regression. and removed S-waiting-on-perf Status: Waiting on a perf run to be completed. labels Apr 10, 2025
bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 10, 2025
[DO NOT MERGE] bootstrap with `-Znext-solver=globally`

A revival of rust-lang#124812.

Current status:

~~`./x.py b --stage 2` passes 🎉~~

`try` builds succeed 🎉 🎉 🎉

[first perf run](rust-lang#133502 (comment)) 👻

### in-flight changes

- rust-lang#124852, unsure whether I actually want to land this PR for now
- rust-lang#139587
- https://github.com/lcnr/rust/tree/opaque-type-method-call

r? `@ghost`
@lcnr lcnr mentioned this pull request Apr 11, 2025
bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 11, 2025
cleanup `mir_borrowck`

Cleanup pulled out of rust-lang#139587. Best reviewed commit by commit.

r? compiler-errors
@lcnr lcnr force-pushed the revealing-use-closures branch 2 times, most recently from 17793b6 to e354bf9 Compare April 11, 2025 11:16
@rust-log-analyzer

This comment has been minimized.

bors added a commit to rust-lang-ci/rust that referenced this pull request Apr 11, 2025
[DO NOT MERGE] bootstrap with `-Znext-solver=globally`

A revival of rust-lang#124812.

Current status:

~~`./x.py b --stage 2` passes 🎉~~

`try` builds succeed 🎉 🎉 🎉

[first perf run](rust-lang#133502 (comment)) 👻

### in-flight changes

- rust-lang#124852, unsure whether I actually want to land this PR for now
- rust-lang#139587
- https://github.com/lcnr/rust/tree/opaque-type-method-call
- rust-lang#138845
- https://gist.github.com/lcnr/86f3e56c3b25a7892d8dbfa28c84e1a8

r? `@ghost`
@lcnr lcnr force-pushed the revealing-use-closures branch 2 times, most recently from b34b851 to 092d8ce Compare April 11, 2025 13:23
@rust-log-analyzer
Copy link
Collaborator

The job mingw-check-tidy failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)
info: removing rustup binaries
info: rustup is uninstalled
##[group]Image checksum input
mingw-check-tidy
# We use the ghcr base image because ghcr doesn't have a rate limit
# and the mingw-check-tidy job doesn't cache docker images in CI.
FROM ghcr.io/rust-lang/ubuntu:22.04

ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends \
  g++ \
---

COPY host-x86_64/mingw-check/validate-toolstate.sh /scripts/
COPY host-x86_64/mingw-check/validate-error-codes.sh /scripts/

# NOTE: intentionally uses python2 for x.py so we can test it still works.
# validate-toolstate only runs in our CI, so it's ok for it to only support python3.
ENV SCRIPT TIDY_PRINT_DIFF=1 python2.7 ../x.py test \
           --stage 0 src/tools/tidy tidyselftest --extra-checks=py,cpp
#
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    pip-compile --allow-unsafe --generate-hashes reuse-requirements.in
---
#12 2.957 Building wheels for collected packages: reuse
#12 2.958   Building wheel for reuse (pyproject.toml): started
#12 3.175   Building wheel for reuse (pyproject.toml): finished with status 'done'
#12 3.176   Created wheel for reuse: filename=reuse-4.0.3-cp310-cp310-manylinux_2_35_x86_64.whl size=132719 sha256=d2a2565e7037ad3883fb9337653f2e25bbb588534fbef3697286cbc26d1bf634
#12 3.176   Stored in directory: /tmp/pip-ephem-wheel-cache-1jmvwu47/wheels/3d/8d/0a/e0fc6aba4494b28a967ab5eaf951c121d9c677958714e34532
#12 3.178 Successfully built reuse
#12 3.179 Installing collected packages: boolean-py, binaryornot, tomlkit, reuse, python-debian, markupsafe, license-expression, jinja2, chardet, attrs
#12 3.585 Successfully installed attrs-23.2.0 binaryornot-0.4.4 boolean-py-4.0 chardet-5.2.0 jinja2-3.1.4 license-expression-30.3.0 markupsafe-2.1.5 python-debian-0.1.49 reuse-4.0.3 tomlkit-0.13.0
#12 3.586 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
#12 4.136 Collecting virtualenv
#12 4.198   Downloading virtualenv-20.31.2-py3-none-any.whl (6.1 MB)
#12 4.434      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.1/6.1 MB 26.1 MB/s eta 0:00:00
#12 4.496 Collecting platformdirs<5,>=3.9.1
#12 4.504   Downloading platformdirs-4.3.8-py3-none-any.whl (18 kB)
#12 4.546 Collecting filelock<4,>=3.12.2
#12 4.554   Downloading filelock-3.18.0-py3-none-any.whl (16 kB)
#12 4.576 Collecting distlib<1,>=0.3.7
#12 4.585   Downloading distlib-0.3.9-py2.py3-none-any.whl (468 kB)
#12 4.595      ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 469.0/469.0 KB 54.2 MB/s eta 0:00:00
#12 4.677 Installing collected packages: distlib, platformdirs, filelock, virtualenv
#12 4.871 Successfully installed distlib-0.3.9 filelock-3.18.0 platformdirs-4.3.8 virtualenv-20.31.2
#12 4.871 WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
#12 DONE 5.0s

#13 [7/8] COPY host-x86_64/mingw-check/validate-toolstate.sh /scripts/
#13 DONE 0.0s
---
DirectMap4k:      139200 kB
DirectMap2M:    10346496 kB
DirectMap1G:     8388608 kB
##[endgroup]
Executing TIDY_PRINT_DIFF=1 python2.7 ../x.py test            --stage 0 src/tools/tidy tidyselftest --extra-checks=py,cpp
+ TIDY_PRINT_DIFF=1 python2.7 ../x.py test --stage 0 src/tools/tidy tidyselftest --extra-checks=py,cpp
##[group]Building bootstrap
    Finished `dev` profile [unoptimized] target(s) in 0.05s
##[endgroup]
WARN: currently no CI rustc builds have rustc debug assertions enabled. Please either set `rust.debug-assertions` to `false` if you want to use download CI rustc or set `rust.download-rustc` to `false`.
[TIMING] core::build_steps::tool::LibcxxVersionTool { target: x86_64-unknown-linux-gnu } -- 0.216
---
fmt check
fmt: checked 6008 files
tidy check
tidy: Skipping binary file check, read-only filesystem
##[error]tidy error: /checkout/compiler/rustc_borrowck/src/region_infer/opaque_types.rs:420: TODO is used for tasks that should be done before merging a PR; If you want to leave a message in the codebase use FIXME
##[error]tidy error: /checkout/compiler/rustc_borrowck/src/region_infer/opaque_types.rs:477: TODO is used for tasks that should be done before merging a PR; If you want to leave a message in the codebase use FIXME
tidy error: /checkout/tests/ui/nll/member-constraints/non-root-universe-existential-1.rs: too many trailing newlines (3)
removing old virtual environment
creating virtual environment at '/checkout/obj/build/venv' using 'python3.10' and 'venv'
creating virtual environment at '/checkout/obj/build/venv' using 'python3.10' and 'virtualenv'
Requirement already satisfied: pip in ./build/venv/lib/python3.10/site-packages (25.1.1)
linting python files
All checks passed!
checking python file formatting
26 files already formatted
checking C++ file formatting
some tidy checks failed
Command has failed. Rerun with -v to see more details.
Build completed unsuccessfully in 0:01:58
  local time: Fri May  9 02:54:15 UTC 2025
  network time: Fri, 09 May 2025 02:54:15 GMT
##[error]Process completed with exit code 1.
Post job cleanup.

@craterbot
Copy link
Collaborator

🚧 Experiment pr-139587-2 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot
Copy link
Collaborator

🎉 Experiment pr-139587-2 is completed!
📊 3 regressed and 0 fixed (4264 total)
📰 Open the full report.

⚠️ If you notice any spurious failure please add them to the denylist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-crater Status: Waiting on a crater run to be completed. labels May 9, 2025
@rustbot
Copy link
Collaborator

rustbot commented May 9, 2025

Requested reviewer is already assigned to this pull request.

Please choose another assignee.

@bors
Copy link
Collaborator

bors commented May 16, 2025

☔ The latest upstream changes (presumably #141066) made this pull request unmergeable. Please resolve the merge conflicts.

tgross35 added a commit to tgross35/rust that referenced this pull request Aug 7, 2025
move `type_check` out of `compute_regions`

A step towards rust-lang#139587. I don't think there's a clear reason for why MIR type check should be in `compute_regions` and this simplifies future PRs here.
Zalathar added a commit to Zalathar/rust that referenced this pull request Aug 8, 2025
move `type_check` out of `compute_regions`

A step towards rust-lang#139587. I don't think there's a clear reason for why MIR type check should be in `compute_regions` and this simplifies future PRs here.
rust-timer added a commit that referenced this pull request Aug 8, 2025
Rollup merge of #145047 - lcnr:yeet-mir-typeck, r=lqd

move `type_check` out of `compute_regions`

A step towards #139587. I don't think there's a clear reason for why MIR type check should be in `compute_regions` and this simplifies future PRs here.
Zalathar added a commit to Zalathar/rust that referenced this pull request Aug 9, 2025
Zalathar added a commit to Zalathar/rust that referenced this pull request Aug 9, 2025
…piler-errors

defer opaque type errors, generally greatly reduce tainting

fixes the test for rust-lang#135528, does not actually fix that issue properly.

This is useful as it causes the migration to rust-lang#139587 to be a lot easier.
github-actions bot pushed a commit to rust-lang/miri that referenced this pull request Aug 9, 2025
move `type_check` out of `compute_regions`

A step towards rust-lang/rust#139587. I don't think there's a clear reason for why MIR type check should be in `compute_regions` and this simplifies future PRs here.
rust-timer added a commit that referenced this pull request Aug 9, 2025
Rollup merge of #145115 - lcnr:less-borrowck-tainting, r=compiler-errors

defer opaque type errors, generally greatly reduce tainting

fixes the test for #135528, does not actually fix that issue properly.

This is useful as it causes the migration to #139587 to be a lot easier.
rust-timer added a commit that referenced this pull request Aug 9, 2025
Rollup merge of #145050 - lcnr:add-opaque-type-tests, r=lqd

add member constraints tests

taken from #139587.
Zalathar added a commit to Zalathar/rust that referenced this pull request Aug 10, 2025
Simplify polonius location-sensitive analysis

This PR reworks the location-sensitive analysis into what we think is a worthwhile subset of the datalog analysis. A sort of polonius alpha analysis that handles NLL problem case 3 and more, but is still using the faster "reachability as an approximation of liveness", as well as the same loans-in-scope computation as NLLs -- and thus doesn't handle full flow-sensitivity like the datalog implementation.

In the last few months, we've identified this subset as being actionable:
- we believe we can make a stabilizable version of this analysis
- it is an improvement over the status quo
- it can also be modeled in a-mir-formality, or some other formalism, for assurances about soundness, and I believe `@nikomatsakis` is interested in looking into this during H2.
- and we've identified the areas of work we wish to explore later to gradually expand the supported cases: the differences between reachability and liveness, support of kills, and considerations of time-traveling, for example.

The approach in this PR is to try less to have the graph only represent live paths, by checking whether we reach a live region during traversal and recording the loan as live there, instead of equating traversal with liveness like today because it has subtleties with the typeck edges in statements (that could forward loans to the successor point without ensuring their liveness). We can then also simplify these typeck stmt edges. And we also can simplify traversal by removing looking at kills, because that's enough to handle a bunch of NLL problem 3 cases -- and we can gradually support them more and more in traversal in the future, to reduce the approximation of liveness.

There's still some in-progress pieces of work w/r/t opaque types that I'm expecting [lcnr's opaque types rework](rust-lang#139587), and [amanda's SCCs rework](rust-lang#130227) to handle. That didn't seem to show up in tests until I rebased today (and shows lack of test coverage once again) when rust-lang#142255 introduced a couple of test failures with the new captures rules from edition 2024. It's not unexpected since we know more work is needed with member constraints (and we're not even using SCCs in this prototype yet)

I'll look into these anyways, both for future work, and checking how these other 2 PRs would change things.

---

I'm not sure the following means a lot until we have some formalism in-place, but:
- I've changed the polonius compare-mode to use this analysis: the tests pass with it, except 2 cases with minor diagnostics differences, and the 2 edition 2024 opaque types one I mentioned above and need to investigate
- things that are expected to work still do work: it bootstraps, can run our rustc-perf benchmarks (and the results are not even that bad), and a crater run didn't find any regressions (forgetting that crater currently fails to test around a quarter of all crates 👼)
- I've added tests with improvements, like the NLL problem case 3 and others, as well as some that behave the same as NLLs today and are thus worse than the datalog implementation

r? `@jackh726`

(no rush I know you're deep in phd work and "implmentating" the new trait solver for r-a :p <3)

This also fixes rust-lang#135646, a diagnostics ICE from the previous implementation.
Zalathar added a commit to Zalathar/rust that referenced this pull request Aug 10, 2025
Simplify polonius location-sensitive analysis

This PR reworks the location-sensitive analysis into what we think is a worthwhile subset of the datalog analysis. A sort of polonius alpha analysis that handles NLL problem case 3 and more, but is still using the faster "reachability as an approximation of liveness", as well as the same loans-in-scope computation as NLLs -- and thus doesn't handle full flow-sensitivity like the datalog implementation.

In the last few months, we've identified this subset as being actionable:
- we believe we can make a stabilizable version of this analysis
- it is an improvement over the status quo
- it can also be modeled in a-mir-formality, or some other formalism, for assurances about soundness, and I believe ``@nikomatsakis`` is interested in looking into this during H2.
- and we've identified the areas of work we wish to explore later to gradually expand the supported cases: the differences between reachability and liveness, support of kills, and considerations of time-traveling, for example.

The approach in this PR is to try less to have the graph only represent live paths, by checking whether we reach a live region during traversal and recording the loan as live there, instead of equating traversal with liveness like today because it has subtleties with the typeck edges in statements (that could forward loans to the successor point without ensuring their liveness). We can then also simplify these typeck stmt edges. And we also can simplify traversal by removing looking at kills, because that's enough to handle a bunch of NLL problem 3 cases -- and we can gradually support them more and more in traversal in the future, to reduce the approximation of liveness.

There's still some in-progress pieces of work w/r/t opaque types that I'm expecting [lcnr's opaque types rework](rust-lang#139587), and [amanda's SCCs rework](rust-lang#130227) to handle. That didn't seem to show up in tests until I rebased today (and shows lack of test coverage once again) when rust-lang#142255 introduced a couple of test failures with the new captures rules from edition 2024. It's not unexpected since we know more work is needed with member constraints (and we're not even using SCCs in this prototype yet)

I'll look into these anyways, both for future work, and checking how these other 2 PRs would change things.

---

I'm not sure the following means a lot until we have some formalism in-place, but:
- I've changed the polonius compare-mode to use this analysis: the tests pass with it, except 2 cases with minor diagnostics differences, and the 2 edition 2024 opaque types one I mentioned above and need to investigate
- things that are expected to work still do work: it bootstraps, can run our rustc-perf benchmarks (and the results are not even that bad), and a crater run didn't find any regressions (forgetting that crater currently fails to test around a quarter of all crates 👼)
- I've added tests with improvements, like the NLL problem case 3 and others, as well as some that behave the same as NLLs today and are thus worse than the datalog implementation

r? ``@jackh726``

(no rush I know you're deep in phd work and "implmentating" the new trait solver for r-a :p <3)

This also fixes rust-lang#135646, a diagnostics ICE from the previous implementation.
Zalathar added a commit to Zalathar/rust that referenced this pull request Aug 10, 2025
Simplify polonius location-sensitive analysis

This PR reworks the location-sensitive analysis into what we think is a worthwhile subset of the datalog analysis. A sort of polonius alpha analysis that handles NLL problem case 3 and more, but is still using the faster "reachability as an approximation of liveness", as well as the same loans-in-scope computation as NLLs -- and thus doesn't handle full flow-sensitivity like the datalog implementation.

In the last few months, we've identified this subset as being actionable:
- we believe we can make a stabilizable version of this analysis
- it is an improvement over the status quo
- it can also be modeled in a-mir-formality, or some other formalism, for assurances about soundness, and I believe ```@nikomatsakis``` is interested in looking into this during H2.
- and we've identified the areas of work we wish to explore later to gradually expand the supported cases: the differences between reachability and liveness, support of kills, and considerations of time-traveling, for example.

The approach in this PR is to try less to have the graph only represent live paths, by checking whether we reach a live region during traversal and recording the loan as live there, instead of equating traversal with liveness like today because it has subtleties with the typeck edges in statements (that could forward loans to the successor point without ensuring their liveness). We can then also simplify these typeck stmt edges. And we also can simplify traversal by removing looking at kills, because that's enough to handle a bunch of NLL problem 3 cases -- and we can gradually support them more and more in traversal in the future, to reduce the approximation of liveness.

There's still some in-progress pieces of work w/r/t opaque types that I'm expecting [lcnr's opaque types rework](rust-lang#139587), and [amanda's SCCs rework](rust-lang#130227) to handle. That didn't seem to show up in tests until I rebased today (and shows lack of test coverage once again) when rust-lang#142255 introduced a couple of test failures with the new captures rules from edition 2024. It's not unexpected since we know more work is needed with member constraints (and we're not even using SCCs in this prototype yet)

I'll look into these anyways, both for future work, and checking how these other 2 PRs would change things.

---

I'm not sure the following means a lot until we have some formalism in-place, but:
- I've changed the polonius compare-mode to use this analysis: the tests pass with it, except 2 cases with minor diagnostics differences, and the 2 edition 2024 opaque types one I mentioned above and need to investigate
- things that are expected to work still do work: it bootstraps, can run our rustc-perf benchmarks (and the results are not even that bad), and a crater run didn't find any regressions (forgetting that crater currently fails to test around a quarter of all crates 👼)
- I've added tests with improvements, like the NLL problem case 3 and others, as well as some that behave the same as NLLs today and are thus worse than the datalog implementation

r? ```@jackh726```

(no rush I know you're deep in phd work and "implmentating" the new trait solver for r-a :p <3)

This also fixes rust-lang#135646, a diagnostics ICE from the previous implementation.
Zalathar added a commit to Zalathar/rust that referenced this pull request Aug 10, 2025
Simplify polonius location-sensitive analysis

This PR reworks the location-sensitive analysis into what we think is a worthwhile subset of the datalog analysis. A sort of polonius alpha analysis that handles NLL problem case 3 and more, but is still using the faster "reachability as an approximation of liveness", as well as the same loans-in-scope computation as NLLs -- and thus doesn't handle full flow-sensitivity like the datalog implementation.

In the last few months, we've identified this subset as being actionable:
- we believe we can make a stabilizable version of this analysis
- it is an improvement over the status quo
- it can also be modeled in a-mir-formality, or some other formalism, for assurances about soundness, and I believe ````@nikomatsakis```` is interested in looking into this during H2.
- and we've identified the areas of work we wish to explore later to gradually expand the supported cases: the differences between reachability and liveness, support of kills, and considerations of time-traveling, for example.

The approach in this PR is to try less to have the graph only represent live paths, by checking whether we reach a live region during traversal and recording the loan as live there, instead of equating traversal with liveness like today because it has subtleties with the typeck edges in statements (that could forward loans to the successor point without ensuring their liveness). We can then also simplify these typeck stmt edges. And we also can simplify traversal by removing looking at kills, because that's enough to handle a bunch of NLL problem 3 cases -- and we can gradually support them more and more in traversal in the future, to reduce the approximation of liveness.

There's still some in-progress pieces of work w/r/t opaque types that I'm expecting [lcnr's opaque types rework](rust-lang#139587), and [amanda's SCCs rework](rust-lang#130227) to handle. That didn't seem to show up in tests until I rebased today (and shows lack of test coverage once again) when rust-lang#142255 introduced a couple of test failures with the new captures rules from edition 2024. It's not unexpected since we know more work is needed with member constraints (and we're not even using SCCs in this prototype yet)

I'll look into these anyways, both for future work, and checking how these other 2 PRs would change things.

---

I'm not sure the following means a lot until we have some formalism in-place, but:
- I've changed the polonius compare-mode to use this analysis: the tests pass with it, except 2 cases with minor diagnostics differences, and the 2 edition 2024 opaque types one I mentioned above and need to investigate
- things that are expected to work still do work: it bootstraps, can run our rustc-perf benchmarks (and the results are not even that bad), and a crater run didn't find any regressions (forgetting that crater currently fails to test around a quarter of all crates 👼)
- I've added tests with improvements, like the NLL problem case 3 and others, as well as some that behave the same as NLLs today and are thus worse than the datalog implementation

r? ````@jackh726````

(no rush I know you're deep in phd work and "implmentating" the new trait solver for r-a :p <3)

This also fixes rust-lang#135646, a diagnostics ICE from the previous implementation.
Zalathar added a commit to Zalathar/rust that referenced this pull request Aug 10, 2025
Simplify polonius location-sensitive analysis

This PR reworks the location-sensitive analysis into what we think is a worthwhile subset of the datalog analysis. A sort of polonius alpha analysis that handles NLL problem case 3 and more, but is still using the faster "reachability as an approximation of liveness", as well as the same loans-in-scope computation as NLLs -- and thus doesn't handle full flow-sensitivity like the datalog implementation.

In the last few months, we've identified this subset as being actionable:
- we believe we can make a stabilizable version of this analysis
- it is an improvement over the status quo
- it can also be modeled in a-mir-formality, or some other formalism, for assurances about soundness, and I believe `````@nikomatsakis````` is interested in looking into this during H2.
- and we've identified the areas of work we wish to explore later to gradually expand the supported cases: the differences between reachability and liveness, support of kills, and considerations of time-traveling, for example.

The approach in this PR is to try less to have the graph only represent live paths, by checking whether we reach a live region during traversal and recording the loan as live there, instead of equating traversal with liveness like today because it has subtleties with the typeck edges in statements (that could forward loans to the successor point without ensuring their liveness). We can then also simplify these typeck stmt edges. And we also can simplify traversal by removing looking at kills, because that's enough to handle a bunch of NLL problem 3 cases -- and we can gradually support them more and more in traversal in the future, to reduce the approximation of liveness.

There's still some in-progress pieces of work w/r/t opaque types that I'm expecting [lcnr's opaque types rework](rust-lang#139587), and [amanda's SCCs rework](rust-lang#130227) to handle. That didn't seem to show up in tests until I rebased today (and shows lack of test coverage once again) when rust-lang#142255 introduced a couple of test failures with the new captures rules from edition 2024. It's not unexpected since we know more work is needed with member constraints (and we're not even using SCCs in this prototype yet)

I'll look into these anyways, both for future work, and checking how these other 2 PRs would change things.

---

I'm not sure the following means a lot until we have some formalism in-place, but:
- I've changed the polonius compare-mode to use this analysis: the tests pass with it, except 2 cases with minor diagnostics differences, and the 2 edition 2024 opaque types one I mentioned above and need to investigate
- things that are expected to work still do work: it bootstraps, can run our rustc-perf benchmarks (and the results are not even that bad), and a crater run didn't find any regressions (forgetting that crater currently fails to test around a quarter of all crates 👼)
- I've added tests with improvements, like the NLL problem case 3 and others, as well as some that behave the same as NLLs today and are thus worse than the datalog implementation

r? `````@jackh726`````

(no rush I know you're deep in phd work and "implmentating" the new trait solver for r-a :p <3)

This also fixes rust-lang#135646, a diagnostics ICE from the previous implementation.
Zalathar added a commit to Zalathar/rust that referenced this pull request Aug 10, 2025
Simplify polonius location-sensitive analysis

This PR reworks the location-sensitive analysis into what we think is a worthwhile subset of the datalog analysis. A sort of polonius alpha analysis that handles NLL problem case 3 and more, but is still using the faster "reachability as an approximation of liveness", as well as the same loans-in-scope computation as NLLs -- and thus doesn't handle full flow-sensitivity like the datalog implementation.

In the last few months, we've identified this subset as being actionable:
- we believe we can make a stabilizable version of this analysis
- it is an improvement over the status quo
- it can also be modeled in a-mir-formality, or some other formalism, for assurances about soundness, and I believe ``````@nikomatsakis`````` is interested in looking into this during H2.
- and we've identified the areas of work we wish to explore later to gradually expand the supported cases: the differences between reachability and liveness, support of kills, and considerations of time-traveling, for example.

The approach in this PR is to try less to have the graph only represent live paths, by checking whether we reach a live region during traversal and recording the loan as live there, instead of equating traversal with liveness like today because it has subtleties with the typeck edges in statements (that could forward loans to the successor point without ensuring their liveness). We can then also simplify these typeck stmt edges. And we also can simplify traversal by removing looking at kills, because that's enough to handle a bunch of NLL problem 3 cases -- and we can gradually support them more and more in traversal in the future, to reduce the approximation of liveness.

There's still some in-progress pieces of work w/r/t opaque types that I'm expecting [lcnr's opaque types rework](rust-lang#139587), and [amanda's SCCs rework](rust-lang#130227) to handle. That didn't seem to show up in tests until I rebased today (and shows lack of test coverage once again) when rust-lang#142255 introduced a couple of test failures with the new captures rules from edition 2024. It's not unexpected since we know more work is needed with member constraints (and we're not even using SCCs in this prototype yet)

I'll look into these anyways, both for future work, and checking how these other 2 PRs would change things.

---

I'm not sure the following means a lot until we have some formalism in-place, but:
- I've changed the polonius compare-mode to use this analysis: the tests pass with it, except 2 cases with minor diagnostics differences, and the 2 edition 2024 opaque types one I mentioned above and need to investigate
- things that are expected to work still do work: it bootstraps, can run our rustc-perf benchmarks (and the results are not even that bad), and a crater run didn't find any regressions (forgetting that crater currently fails to test around a quarter of all crates 👼)
- I've added tests with improvements, like the NLL problem case 3 and others, as well as some that behave the same as NLLs today and are thus worse than the datalog implementation

r? ``````@jackh726``````

(no rush I know you're deep in phd work and "implmentating" the new trait solver for r-a :p <3)

This also fixes rust-lang#135646, a diagnostics ICE from the previous implementation.
Zalathar added a commit to Zalathar/rust that referenced this pull request Aug 10, 2025
Simplify polonius location-sensitive analysis

This PR reworks the location-sensitive analysis into what we think is a worthwhile subset of the datalog analysis. A sort of polonius alpha analysis that handles NLL problem case 3 and more, but is still using the faster "reachability as an approximation of liveness", as well as the same loans-in-scope computation as NLLs -- and thus doesn't handle full flow-sensitivity like the datalog implementation.

In the last few months, we've identified this subset as being actionable:
- we believe we can make a stabilizable version of this analysis
- it is an improvement over the status quo
- it can also be modeled in a-mir-formality, or some other formalism, for assurances about soundness, and I believe ```````@nikomatsakis``````` is interested in looking into this during H2.
- and we've identified the areas of work we wish to explore later to gradually expand the supported cases: the differences between reachability and liveness, support of kills, and considerations of time-traveling, for example.

The approach in this PR is to try less to have the graph only represent live paths, by checking whether we reach a live region during traversal and recording the loan as live there, instead of equating traversal with liveness like today because it has subtleties with the typeck edges in statements (that could forward loans to the successor point without ensuring their liveness). We can then also simplify these typeck stmt edges. And we also can simplify traversal by removing looking at kills, because that's enough to handle a bunch of NLL problem 3 cases -- and we can gradually support them more and more in traversal in the future, to reduce the approximation of liveness.

There's still some in-progress pieces of work w/r/t opaque types that I'm expecting [lcnr's opaque types rework](rust-lang#139587), and [amanda's SCCs rework](rust-lang#130227) to handle. That didn't seem to show up in tests until I rebased today (and shows lack of test coverage once again) when rust-lang#142255 introduced a couple of test failures with the new captures rules from edition 2024. It's not unexpected since we know more work is needed with member constraints (and we're not even using SCCs in this prototype yet)

I'll look into these anyways, both for future work, and checking how these other 2 PRs would change things.

---

I'm not sure the following means a lot until we have some formalism in-place, but:
- I've changed the polonius compare-mode to use this analysis: the tests pass with it, except 2 cases with minor diagnostics differences, and the 2 edition 2024 opaque types one I mentioned above and need to investigate
- things that are expected to work still do work: it bootstraps, can run our rustc-perf benchmarks (and the results are not even that bad), and a crater run didn't find any regressions (forgetting that crater currently fails to test around a quarter of all crates 👼)
- I've added tests with improvements, like the NLL problem case 3 and others, as well as some that behave the same as NLLs today and are thus worse than the datalog implementation

r? ```````@jackh726```````

(no rush I know you're deep in phd work and "implmentating" the new trait solver for r-a :p <3)

This also fixes rust-lang#135646, a diagnostics ICE from the previous implementation.
Zalathar added a commit to Zalathar/rust that referenced this pull request Aug 10, 2025
Simplify polonius location-sensitive analysis

This PR reworks the location-sensitive analysis into what we think is a worthwhile subset of the datalog analysis. A sort of polonius alpha analysis that handles NLL problem case 3 and more, but is still using the faster "reachability as an approximation of liveness", as well as the same loans-in-scope computation as NLLs -- and thus doesn't handle full flow-sensitivity like the datalog implementation.

In the last few months, we've identified this subset as being actionable:
- we believe we can make a stabilizable version of this analysis
- it is an improvement over the status quo
- it can also be modeled in a-mir-formality, or some other formalism, for assurances about soundness, and I believe ````````@nikomatsakis```````` is interested in looking into this during H2.
- and we've identified the areas of work we wish to explore later to gradually expand the supported cases: the differences between reachability and liveness, support of kills, and considerations of time-traveling, for example.

The approach in this PR is to try less to have the graph only represent live paths, by checking whether we reach a live region during traversal and recording the loan as live there, instead of equating traversal with liveness like today because it has subtleties with the typeck edges in statements (that could forward loans to the successor point without ensuring their liveness). We can then also simplify these typeck stmt edges. And we also can simplify traversal by removing looking at kills, because that's enough to handle a bunch of NLL problem 3 cases -- and we can gradually support them more and more in traversal in the future, to reduce the approximation of liveness.

There's still some in-progress pieces of work w/r/t opaque types that I'm expecting [lcnr's opaque types rework](rust-lang#139587), and [amanda's SCCs rework](rust-lang#130227) to handle. That didn't seem to show up in tests until I rebased today (and shows lack of test coverage once again) when rust-lang#142255 introduced a couple of test failures with the new captures rules from edition 2024. It's not unexpected since we know more work is needed with member constraints (and we're not even using SCCs in this prototype yet)

I'll look into these anyways, both for future work, and checking how these other 2 PRs would change things.

---

I'm not sure the following means a lot until we have some formalism in-place, but:
- I've changed the polonius compare-mode to use this analysis: the tests pass with it, except 2 cases with minor diagnostics differences, and the 2 edition 2024 opaque types one I mentioned above and need to investigate
- things that are expected to work still do work: it bootstraps, can run our rustc-perf benchmarks (and the results are not even that bad), and a crater run didn't find any regressions (forgetting that crater currently fails to test around a quarter of all crates 👼)
- I've added tests with improvements, like the NLL problem case 3 and others, as well as some that behave the same as NLLs today and are thus worse than the datalog implementation

r? ````````@jackh726````````

(no rush I know you're deep in phd work and "implmentating" the new trait solver for r-a :p <3)

This also fixes rust-lang#135646, a diagnostics ICE from the previous implementation.
rust-timer added a commit that referenced this pull request Aug 10, 2025
Rollup merge of #143093 - lqd:polonius-pre-alpha, r=jackh726

Simplify polonius location-sensitive analysis

This PR reworks the location-sensitive analysis into what we think is a worthwhile subset of the datalog analysis. A sort of polonius alpha analysis that handles NLL problem case 3 and more, but is still using the faster "reachability as an approximation of liveness", as well as the same loans-in-scope computation as NLLs -- and thus doesn't handle full flow-sensitivity like the datalog implementation.

In the last few months, we've identified this subset as being actionable:
- we believe we can make a stabilizable version of this analysis
- it is an improvement over the status quo
- it can also be modeled in a-mir-formality, or some other formalism, for assurances about soundness, and I believe ````````@nikomatsakis```````` is interested in looking into this during H2.
- and we've identified the areas of work we wish to explore later to gradually expand the supported cases: the differences between reachability and liveness, support of kills, and considerations of time-traveling, for example.

The approach in this PR is to try less to have the graph only represent live paths, by checking whether we reach a live region during traversal and recording the loan as live there, instead of equating traversal with liveness like today because it has subtleties with the typeck edges in statements (that could forward loans to the successor point without ensuring their liveness). We can then also simplify these typeck stmt edges. And we also can simplify traversal by removing looking at kills, because that's enough to handle a bunch of NLL problem 3 cases -- and we can gradually support them more and more in traversal in the future, to reduce the approximation of liveness.

There's still some in-progress pieces of work w/r/t opaque types that I'm expecting [lcnr's opaque types rework](#139587), and [amanda's SCCs rework](#130227) to handle. That didn't seem to show up in tests until I rebased today (and shows lack of test coverage once again) when #142255 introduced a couple of test failures with the new captures rules from edition 2024. It's not unexpected since we know more work is needed with member constraints (and we're not even using SCCs in this prototype yet)

I'll look into these anyways, both for future work, and checking how these other 2 PRs would change things.

---

I'm not sure the following means a lot until we have some formalism in-place, but:
- I've changed the polonius compare-mode to use this analysis: the tests pass with it, except 2 cases with minor diagnostics differences, and the 2 edition 2024 opaque types one I mentioned above and need to investigate
- things that are expected to work still do work: it bootstraps, can run our rustc-perf benchmarks (and the results are not even that bad), and a crater run didn't find any regressions (forgetting that crater currently fails to test around a quarter of all crates 👼)
- I've added tests with improvements, like the NLL problem case 3 and others, as well as some that behave the same as NLLs today and are thus worse than the datalog implementation

r? ````````@jackh726````````

(no rush I know you're deep in phd work and "implmentating" the new trait solver for r-a :p <3)

This also fixes #135646, a diagnostics ICE from the previous implementation.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
perf-regression Performance regression. S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. WG-trait-system-refactor The Rustc Trait System Refactor Initiative (-Znext-solver)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants